The present invention relates generally to information analysis systems and methods, and more particularly to methods and systems for processing and analyzing streaming data.
In a system that encounters streaming data that has to be acted upon over time, the data is typically stored in the order in which it first arrived and then processed again and again over time as more data enters the system. The problem is that each time processing occurs, the data must be retrieved and rehandled, causing processing to get slower as the system acquires more and more data. It is an object of the present invention to address this shortcoming by providing a method and system that captures streaming data and avoids continual retrieval and rehandling. It is a further object of the present invention to provide a method and system which, in some cases, can become faster as the system acquires more data.
Briefly described, an information analysis system for processing and analyzing streaming data includes an engine processor having a plurality of inputs, a plurality of separate outputs, and a plurality of separate bi-directional input/output terminals. The engine processor is operative to process data according to stored information. A rules storage module is coupled to one input of said engine processor and operative to store a set of stored rules in defining engine operation. An input events module is coupled to another input of said engine processor and operative for receiving events and for providing a plurality of events from at least one received event, wherein the engine processes the events according to the stored rules for generating at a first output a series of valid results when predetermined input events occur according to the rules as processed by the engine. A first state storage unit is coupled to a bi-directional input/output terminal of the engine and is operative to store a state of a rule that the engine has matched to an input event but which has not yet produced a valid result, where the state of a rule is stored enabling the engine to provide a valid result output upon a future reception of an input event necessary to complete the rule. A transaction log module is coupled to an output of the engine and is operative to monitor the operation of the engine for specified time period necessary to complete partially satisfied rules.
The invention also includes a method for automatically triggering output events based upon a plurality of input events and a plurality of rules, wherein each of the rules can have a partially satisfied state. The method includes receiving at least one input event; restoring at least one state associated with the input event, the state indicative of ones of the rules having a partially satisfied state; applying the rules to the input event in light of the restored state to determine if any of the rules is fully satisfied, and if so triggering an output event based upon the fully satisfied rules; and, storing an updated state dependently the at least one other event.